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[57] ABSTRACT 

A feedback apparatus for an operator to control an effector 
that is remote from the operator to interact with a remote 
environment has a local input device to be manipulated by 
the operator. Sensors in the effector’s environment are 
capable of sensing the amplitude of forces arising between 
the effector and its environment, the direction of application 
of such forces, or both amplitude and direction. A feedback 
signal corresponding to such a component of the force, is 
generated and transmitted to the environment of the opera- 
tor. The signal is transduced into a vibrotactile sensory 
substitution signal to which the operator is sensitive. Vibra- 
tion producing apparatus present the vibrotactile signal to 
the operator. The full range of the force amplitude may be 
represented by a single, mechanical vibrator. Vibrotactile 
display elements can be located on the operator’s limbs, 
such as on the hand, fingers, arms, legs, feet, etc. The 
location of the application of the force may also be specified 
by the location of a vibrotactile display on the operator’s 
body. Alternatively, the location may be specified by the 
frequency of a vibrotactile signal. 

24 Claims, 5 Drawing Sheets 
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APPARATUS FOR PROVIDING 
VIBROTACTILE SENSORY SUBSTITUTION 
OF FORCE FEEDBACK 

U.S. GOVERNMENT INTEREST 

The U.S. Government has certain rights in this invention 
pursuant to Contract No. NGT-50145, awarded by NASA 
and contract/Grant no. 956892 awarded by California Insti- 
tute of Technology get Propulsion Lab. 

This is a continuation of application Ser. No. 08/004,761 
filed on Jan. 14, 1993 now U.S. Pat. No. 5,451,924. 

The invention relates in general to the field of non- visual, 
non-reactive displays of information. More particularly, the 
invention relates to the display of information, such as force 
feedback, using non reactive sensory substitution displays, 
such as vibrotactile and auditory displays. 

BACKGROUND 

For many operations, a human operator manipulates an 
input device in an environment local to the operator, and the 
effect of the manipulation takes place in a remote environ- 
ment, by virtue of the actions of an effector located in the 
remote environment. Such operations include the manipu- 
lation of effectors: in outer space by ground based operators; 
in hazardous environments, such as a nuclear reactor core, 
by operators a safe distance away; attached to construction 
equipment, such as cranes by operators distant from the 
crane. To facilitate the control of the effector, various forms 
of feedback have been used. Visual feedback, typically using 
a video camera and visual display device is very helpful. 
However, in many situations, visual feedback is not avail- 
able, such as when the remote environment is dark or murky, 
for instance below water. Visual feedback can be degraded 
due to the various complexities involved in video transmis- 
sion. Further, video transmission requires a rather large 
bandwidth and relatively expensive equipment. Also, due to 
the arrangement of the effector with respect to the remote 
environment, the view of the visual capture device, such as 
a camera, is often blocked by intervening elements. 

It is also common to provide a physically reactive force 
feedback. In such an arrangement, the effector is provided 
with an array of force sensors, which are designed and 
located to measure the forces applied to the effector by 
interaction with its environment. These forces are trans- 
duced and transmitted to the environment of the local 
operator, where they are retransduced by motor and gear 
systems into forces that are applied to the input apparatus 
that the operator uses. Thus, the operator “feels” the forces 
that the effector feels, to some degree depending on the 
fidelity of the force sensing and reproduction scheme. 

Force information is useful for several reasons. It facili- 
tates governing the forces applied by the effector, either to 
avoid excessive stresses or to produce a desired movement 
of an object, or to minimize energy expenditure. It also 
facilitates estimating mass, frictional resistance, or other 
properties of objects in the remote environment in order to 
predict their behavior. It also provides information about 
contact with objects or with features of the environment, as 
in searching when visual information is unavailable. In some 
cases, it also guides the operator’s body (typically the hand) 
to an appropriate location. 

Use of reactive force feedback has drawbacks. The appa- 
ratus required to reproduce the force at the local environ- 
ment of the operator may be costly, complicated and bulky. 
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Further, the force fed back to the operator’s hand acts as an 
additional input to the input device, thereby having an effect 
on the remote effector. It is also often desirable to amplify 
the force feedback signal, to enable sensing modest or 
5 delicate forces. If the gain on the feedback signal is large, in 
order to provide needed sensitivity, a small force at a remote 
location would be presented as a large force to the operator. 
This would impose a large force on the operator and could 
make the input controller more difficult to move. Thus the 
10 task would be made more physically difficult, the operation 
would be slowed down, and the rate of operator fatigue 
would be increased. If gains are made excessively high, 
instability could arise. 

Some force feedback systems present the feedback with a 
15 time delay. Time delays on the order of several seconds are 
common in connection with communication to outer space, 
even with the signal being transmitted at the speed of radio 
waves. It has generally been found that reactive feedback in 
the presence of time delays can not be used effectively 
20 because instabilities quickly arise. See generally, W. R. 
Ferrel, “Delayed Force Feedback,: Human Factors , October 
1986, pp. 449-455. 

It has been noted that if reactive force feedback is 
provided to the operator, not to the hand that is applying the 
25 force, but to the other hand, the operator gains an advantage. 
See S. Weissenberger and T. B. Sheridan, “Dynamics of 
human operator control systems using tactile feedback,” 
A.SM.E. Transactions (J. Basic Eng.), vol. 84, pp. 297-301 
(1962). However, the fact that the feedback is provided to 
30 another limb, and, in fact, an oppositely disposed limb, 
leaves room for improvement. 

It has also been attempted to provide force feedback 
through the visual modality, such as by using a visual 
35 display that indicates force, either through numbers, or a 
pictorial representation. However, the visual modality is 
typically heavily burdened by other demands. It has been 
found that such methods typically overload the operator’s 
ability to process visually perceived information, thus lead- 
40 ing to degraded or unacceptable performance. 

There are other situations where feedback of force is 
desirable, which are not obviously within the class of remote 
effector applications. For instance, in microsurgery, or any 
micromanipulation, the operator manipulates an input 
45 device with a resultant motion by an effector, perhaps very 
close to the operator, yet connected by a linkage (mechani- 
cal, electrical, electromechanical, or other) that effectively 
distances the operator from the resultant activity as if the 
operator were far away. Such microsurgery could be con- 
50 ducted over distances of either inches or miles, with the 
appropriate visual feedback tools. Another instance relates 
to the control of artificial limbs, for instance an artificial arm 
or hand. One type of artificial limb is referred to as myeo- 
electric. Such a device is connected to the receive impulses 
55 from the nerves of the muscles remaining in the stump of the 
limb. The patient still has the ability to move these muscles, 
thereby generating electric signals, which can be used as 
inputs to the prosthesis. It would also be helpful if the patient 
were aware of the force field in which the prosthesis fingers, 
6o hand, wrist, elbow, etc. operates. Force related cues could 
provide valuable information to the patient. However, reac- 
tive force feedback is not practical in such a situation 
because the user has no sensation in the artificial limb. 

Previous work of the present inventor has indicated that a 
65 vibrotactile display can be used to indicate whether or not a 
remote effector has come into contact with an object in the 
remote environment. In that work, a sensor/transducer car- 
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ried by an effector was connected to a computer, which 
generated a signal to a vibrating unit fixed to the hand of the 
user that guided the input device. The computer generated 
signal was not related in any way to the magnitude of the 
force experienced by the sensor, other than that the signal 5 
indicated a positive force if the force exceeded the minimal 
sensitivity threshold of the sensor. No information was 
provided to the user about the magnitude or the direction of 
the force. See N. J. M. Patrick “Design, Construction, and 
Testing of a Fingertip Tactile Display for Interaction with 10 
Virtual and Remote Environments,” submitted in partial 
fulfillment of the requirements for the degree of Master of 
Science in Mechanical Engineering at the Massachusetts 
Institute of Technology pp. 2 and 63-67 (1990), citing the 
present inventor’s unpublished research. The entire docu- 15 
ment is hereby incorporated fully herein by reference. The 
principal focus of the Patrick work was using a vibrotactile 
display to track the location of the user’s fingers relative to 
desired locations, particularly in the context of a virtual 
reality glove manipulation. The signal generated by the 20 
display device was a vibration having an amplitude linearly 
related to the distance between the location of the fingers and 
the desired location. 
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In connection with the foregoing embodiment, vibrotac- 
tile display elements can be located on the operator’s limbs, 
such as on the hand, fingers, arms, legs, feet, etc. Auditory 
display elements may be stereo headphones or free standing 
audio speakers, numbering from one to many more than two. 

According to another preferred embodiment, the inven- 
tion includes an apparatus to generate a feedback signal such 
as discussed above, that corresponds to the location, relative 
to the effector and its environment, of the force between the 
effector and its environment. The location may be specified 
by the location of a vibrotactile display on the operator’s 
body, or the location of audio speakers that generate signals 
corresponding to specific forces. Alternatively, the location 
may be specified by the frequency of an audio signal, or by 
the apparent location of an audio signal, as simulated by a 
combination of signals originating at different locations. 

Another preferred embodiment of the invention is a 
system, incorporating any of the feedback apparatus as 
discussed above, further including an effector and an input 
device. Additionally, the system may include a visual feed- 
back system, such as a video camera at the effector’s 
location and a playback device at the operator’s location. 


OBJECTS 

An object of the invention is thus to provide a non- visual, 
non-reactive sensory substitution force feedback signal for 
use in connection with remote effector applications, which 
does not overload the capabilities of the operator and which 
remains stable even in the presence of time delays. A further 
object is to provide the advantages of force feedback without 
the burdens of reactive force feedback, such as an expensive 
and heavy or complicated effector and input device and 
instability in the presence of time delays. Another object of 
the invention is to facilitate increasing the gain of a force 
feedback system, in connection with the sensing of very 
small forces, without unduly increasing the resistance of the 
input device to operator control. Yet another object of the 
invention is to provide force feedback that is relatively 
inexpensive and robust and requires a minimal bandwidth to 
transmit the signal from the remote location to the local 
environment. 


SUMMARY 

A preferred embodiment of the invention is a feedback 
apparatus for an operator to control an effector that is remote 
from the operator to interact with a remote environment. The 50 
operator manipulates a local input device. The feedback 
apparatus includes sensors in the effector’s environment 
capable of sensing the amplitude of forces arising between 
the effector and its environment, the direction of application 
of such forces, or both amplitude and direction. Means are 55 
provided for generating a feedback signal corresponding to 
such a component of the force, transmitting the feedback 
signal to the environment of the operator and for transducing 
the signal into a non-visual, non-reactive sensory substitu- 
tion signal to which the operator is sensitive. For example, 60 
the sensory substitution signal may be auditory or vibrotac- 
tile. The invention also includes means for presenting the 
sensory substitution signal to a sensory system of the 
operator that is receptive to the signal. The full range of the 
force amplitude may be represented by a single, unitary 65 
display means, such as a mechanical vibrator or an audio 
speaker. 


BRIEF DESCRIPTION OF THE FIGURES 

These and other features, aspects, and advantages of the 
present invention will become better understood with regard 
to the following description, appended claims and accom- 
panying drawings, where: 

FIG. 1 is a Schematic representation of an embodiment of 
the invention using a vibrotactile display. 

FIG. 2A is a schematic representation of a force sensing 
resistor that may be used with the invention in plan view. 

FIG. 2B is a side cross-sectional view of the force sensing 
resistor shown in FIG. 2A. 

FIG. 3 is a schematic representation of a number of force 
sensing resistors arrayed on a portion of an effector. 

FIG. 4 is a schematic representation of an electronic 
circuit that can be used to amplify the signal generated by a 
force sensing resistor. 

FIG. 5 is a schematic representation of an embodiment of 
the invention using an auditory display. 

FIG. 6 is a schematic representation of a number of force 
sensing resistors arrayed on another effector. 

FIG. 7 is a schematic representation of a number of 
vibrotactile display element is arrayed on an operator’s 
hand. 

FIG. 8 is a schematic representation showing the local 
environment of an operator and the distant environment of 
the effector, with a visual feedback mechanism and means 
for controlling the effector and sensors. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENTS OF THE 
INVENTION 

In a preferred embodiment, the invention is an apparatus 
for providing force feedback through a sensory substitution 
display, for instance a vibrotactile display with a change in 
the amplitude of the vibration representing a change in the 
magnitude of the force. A representative embodiment of the 
invention is shown schematically in FIG. 1. A number of 
force sensors 106 1 - 106 m are used, the number depending on 
the effector, the environment and the specificity of the 
information required by the task at hand. Suitable force 
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sensors are described below. For each force sensor 106, an 
analog to digital converter (A/D) 104 1 _ m is provided. For 
instance, the A/D convertors can be implemented by a Data 
Translation DT-2801 input-output (I/O) board with sixteen 
analog to digital (A/D) channels and two digital to analog 5 
(D/A) channels. The board is installed in a programmable 
general purpose digital computer processor, such as an 
AT&T 6300 Personal Computer, which is used to process 
and thus augment the digital signals generated by the A/D 
converters 104. A number of digital to analog converters 
lOSj.^ are connected to the output of the processor 102. 
Some of the D/A converters 108 may be implemented by an 
output board such as a Data Translation DT-2816 output 
board. The number of A/D channels used for a given 
application correspond to the number of force sensors 
needed for the task. The number of D/A channels used 15 
depends on the number of display elements 110. There need 
not be the same number of D/A channels as A/D channels, 
or as individual sensor signals. 

The D/A channels 108 on the DT-2801 and DT-2816 
output boards stimulate a number of vibrators 110 j_ x that 20 
together constitute the vibrotactile display 112 and present 
the vibrotactile stimuli to the operator. The vibrators are 
attached to selected parts of the operator’s body, as 
described below. 

Suitable force sensors are force sensing resistors (“fsr’s”), 25 
as shown schematically in FIGS. 2 A and 2B, such as made 
by Interlink Electronics of Santa Barbara Calif. As described 
in the Product Design Guide, available from Interlink Elec- 
tronics, the fsr 106 is a polymer thick film device which 
exhibits a decreasing resistance with increasing force 30 
applied normal to the device surface. A basic fsr has two 
polymer sheets laminated together: one sheet 206 is coated 
with interdigitating electrodes 202 and 204. The other sheet 
208 is coated with a proprietary semiconductive material. 
When a force is applied to the fsr 106, the semiconductive 35 
material shunts the interdigitating electrodes, causing the 
resistance to drop. Such resistors are described more fully at 
“Touch the Future Through Force Sensing Resistors from 
Interlink Electronics,” also available from Interlink Elec- 
tronics, which is incorporated fully herein by reference. 40 
Fsr’s are available in a variety of shapes and sizes. A typical 
range is an area of from 0.5x0.5 cm (0.2x0.2 in.) to 56x80 
cm. (22x32 in.). Virtually any shape desirable is available, 
including squares, circles, etc. The sensitivity of such sen- 
sors ranges from ten grams to twenty kilograms. 40 

The fsr’s described above are useful for sensing forces 
that are normal to the surface to which the sensor is attached. 

For instance, as shown in FIG. 3, if such a sensor 3061 were 
applied to the tip of a fingerlike effector 302, it would be 5Q 
sensitive to forces experienced by the effector when pushing 
parallel to its long axis against an object 308, analogous to 
a human finger pushing a button. It is also possible to use 
other types of sensors 306 2 , such as strain gauges, mounted 
along the side of such a finger-like effector, to sense and thus 55 
transduce bending stresses in the effector arising due to 
forces applied normal to the long axis of the effector, for 
instance parallel to arrows B t and B^ 

It is necessary to provide an electric circuit that transduces 
the voltage output from the fsr’s 106 into a usefully cali- 60 
brated voltage spectrum. For example, a typical effector may 
be able to apply forces to the environment ranging from zero 
to ten lbs-f. With a current through the fsr limited to 
approximately 1 to 2 mA, an appropriate circuit is shown in 
FIG. 4. 65 

The power supply provides +12 volts and ground termi- 
nals. (A -12 V terminal may also be provided.) The resistors 
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414 and 412, a trim potentiometer serve as a +10 volt power 
source to the positive input terminal of the op-amp 410. To 
accomplish this the voltage into the op-amp is measured 
from ground up to the trim-potentiometer, and the trim- 
potentiometer is adjusted until the voltage is exactly 10 
volts. 

The properties of op-amps dictate that the voltage at the 
negative terminal is +10 V, since the voltage at the positive 
terminal is set to be +10 V. The voltage across the fsr 406 is 
then 2 V. The resistance of each fsr may range, for example, 
from over 1 M-ohm at zero force to a smaller resistance at 
maximum force. For a typical fsr the minimum resistance is 
approximately 1500 ohms at five pounds of force. Thus the 
current through the fsr can range from approximately 0 A at 
zero force to approximately 1.3 mA at high force, which is 
within the suggested range for fsr current. 

For such an fsr 406, the circuit having a 4.7 k-ohm resistor 
402 in series with a 0-5 k-ohm trim potentiometer 408 as 
shown in the upper part of the circuit diagram, allows 
voltage swings at V oul from 10 V when no force is present 
to 0 V when a five pound force is present. These resistance 
and trim potentiometer magnitudes are dependent on the 
characteristics of each individual fsr 406. The voltage equa- 
tion for the upper portion of the circuit is 

nv=v fsr w^v 2 +v out ( 1 ) 

where W fsr is the voltage across the fsr, which equals 2 V; V L 
is the voltage across resistor 402, V 2 is the voltage across the 
in loop (left) portion of the potentiometer 408; and V OM/ is 
the op-amp output voltage, which corresponds to the voltage 
across the portion of the trim potentiometer 408 not con- 
nected in series with the resistor 402. The above equation (1) 
then reduces to 

v 1+ v 2 +v ou r=lO v (2) 

Since an op- amp draws no current, the current through the 
resistors 406, 402 and 408 equals 

(V fs /R /sr M2 v/R fsr ) (3) 

where R fsr is the resistance of the fsr 406 measured at any 
particular time. The equation (3) reduces to 

V ou f= 10 V— (4.7 k-ohm) (2 V/R /sr )-(R 2 ) (2 V/R fsr ). (4) 

When there is no force at the fsr 406, R /Jr is very large and 
W out equals ten volts. The correct value to which the trim 
potentiometer 408 is adjusted depends on the characteristics 
of the particular fsr 406. The fsr resistance value at maxi- 
mum force is substituted for R /vr and V oul is set to be zero. 
The trim potentiometer 408 is tuned to insure that W out is 
zero volts at maximum force, giving a swing of 10 V to zero 
V for changes in force from zero pounds to maximum force. 
For example, if R fsr were 1400 k-ohm at maximum force, the 
trim potentiometer 408 resistance in series with the 4.7 
k-ohm resistor 402 would be set to approximately 2.3 
k-ohm, giving a V out of zero at maximum force. 

For a typical display, the number of fsrs used may range 
from two to nine, however, even more sensors can be used. 
A suitable vibrotactile display electronics unit is described in 
detail by Patrick, cited above. Briefly, the vibrotactile dis- 
play, consists o vibrating voice coils. Suitable vibrators are 
available from Audiological Engineering Corporation in 
Somerville, Mass. The displays vibrate at 250 Hz, which is 
the frequency at which the Pacinian corpuscles (the rapidly 
adapting vibration detectors in the skin) have their lowest 
threshold. The resonant frequency of the vibrators is also 
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250 Hz. Therefore the highest efficiency of voltage into the 
display and vibration amplitude is achieved. Suitable cir- 
cuitry is provided as part of vibrotactile display 112 to 
convert the voltage output from D/A convertors 108 into a 
periodic signal suitable for excitation of vibrating elements 5 
110. Design of such a circuit is within the competence of one 
of ordinary skill in the art and is described fully in Patrick, 
cited above. 

As shown schematically in, FIG. 7, the vibrotactile dis- 
plays may be placed on various body locations, such as on 
the fingertips 710J and 710 2 , palm of the hand 710 3 , and 
wrist 710 4 , depending on the number and location of critical 
forces present for an experimental task. A simple joystick 
type input device 720 is shown in FIG. 7, however, more 
sophisticated input devices can be used, such as a joystick in 15 
combination with a pair of spring loaded arms or a virtual 
reality type glove, such as are available from VPL Research 
of California, under the tradename “Dataglove.” 

Placement of the vibrating elements on the fingertips and 
palms is for two reasons: 1 ) they are the regions of the skin 2Q 
which are most sensitive to vibrotactile stimulation (see 
Wilska, A. (1954). “On the Vibrational Sensitivity in Dif- 
ferent Regions of the Body Surface,” Acta Physiologica 
Scandinavica , Vol. 31, pp. 285-289) and 2) they allow the 
displays to be placed in positions which allowed easy 25 
transformations through sensory substitution for the opera- 
tor to identify the position of the forces exerted at the remote 
effector location. The vibrators may be held in place on the 
skin using nylon hook and loop fasteners. 

According to a preferred embodiment, the magnitude of 3Q 
vibration depends on the magnitude of the force being 
displayed by the vibrating element. Research studies have 
determined relationships for the perceived magnitude of a 
vibrotactile stimulus as a function of the physical intensity 
of the stimulus. The physical intensity is the displacement of 35 
the vibrator which is usually measured in microns. A study 
by Verrillo, R. T. and Chamberlain, S. C. , “The effect of 
Neural Density and Contactor Surround on Vibrotactile 
Sensation Magnitude.,” Perception and Psychophysics , Vol. 

11 (IB), pp. 117-120 (1972) (which is incorporated fully 4Q 
herein by reference) shows a relationship between displace- 
ment and sensation magnitude according to the function: 

{SMf A *=k*D, (4) 

where SM is sensation magnitude, and D is displacement in 45 
db and k is a constant determined by measurement. Db’s are 
related to displacement in microns by: 

db = 20 log (P), (5) 

50 

where P is the displacement in microns. The relationship 
between voltage in and displacement P can be calibrated. 
Converting displacement in db to displacement in microns, 
applying the V in to displacement function, and linearly 
scaling subjective magnitude between 0 and 5 pounds for x 55 
and z-directions and between 0 an 10 pounds for the 
y-direction, gives the following relationships between force 
and input voltage: 

V i .„=0.0493*10°- 78Vo ^°' 48 -0.0493 (6) ^ 

60 

for forces in x and z-directions and 

^=0.0493*10° 536 * /oree ° 48 -0.0493 (7) 

for forces in the y-direction. The (-0.0493) is included so 65 
that zero force would correspond to zero volts. The rela- 
tionship allow a change in force to be perceived as a change 
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in the magnitude of the vibrotactile stimulus and are used in 
software routines that control the processing by the proces- 
sor 102. The software is relatively straightforward and is 
well within the scope of the skill of one familiar with the art. 

The specific conformation of the implementation of the 
invention depends on the task to be accomplished, which in 
turn governs the shape, degrees of freedom and capabilities 
of the effector as well as the control mechanism. For 
instance, a typical effector is shown schematically in FIG. 6 . 
The effector has many degrees of freedom (“dof ’) plus a 
gripping capability. Three of the degrees of freedom, des- 
ignated X, Y and Z, identify arm translation. One (“Azi- 
muth”) identifies arm rotation. Another (“Elevate”) identi- 
fies gripper rotary elevation and another (“Twist”) specifies 
gripper twist. Jaw grasping is indicated at “Jaw.” The arm 
can also rotate about the X, Y and Z axes. 

Force sensing resistors may be placed at various locations 
on the effector. For instance, as shown schematically in FIG. 
6 , sensors 606 1 and 606 2 are located on the inside, or 
grasping surfaces of the jaw members. Such sensors would 
provide information about the force of grasping, such as for 
holding delicate objects. Sensors 606 5 and 606 6 are located 
on the outside surfaces of the jaw members. Such sensors 
would provide information concerning the jaw contacting 
the environment, or the amount of spreading forces that the 
jaw members may be providing. Sensors 606 3 and 606 4 are 
located on the end surfaces of the jaw members. These 
sensors would provide information concerning the force 
with which the gripper may be pushing against an object in 
the X direction directly ahead of it. Sensors 606 7 , 606 8 and 
606 9 are located on the arm of the effector. These sensors 
may be a different sort of sensor, such as a strain gauge, that 
measure bending strains in the arm. They may provide 
information concerning the downward force being applied to 
the arm, such as pertaining to the weight of an object being 
grasped by the jaws. 

Signals from the individual sensors may combine to 
provide information pertaining to the direction of an applied 
force. For instance, if both sensors 606 3 and 606 5 indicate a 
force of certain respective magnitudes, an operator might 
conclude that the gripper member was pushing against a 
comer, or portion of the environment having two surfaces at 
an angle to each other. The amount of information that the 
sensors can provide depends to some extent on the famil- 
iarity the operator has with the environment in which the 
effector operates. For instance, if an effector is operating in 
an environment of known configuration, but having some 
portions for which available visual feedback is obstructed, 
much information can be gained from knowing which 
portions of the effector are contacting the environment. Such 
situations might include locations associated with a nuclear 
reactor, or certain medical situations where other imaging 
has already provided a “geography” of the locus of interest. 

The manner in which signals from the sensors are pre- 
sented to the operator also depends on the situation. The 
system should be designed so that the operator can associate 
the stimulus signals with the environment they simulate. For 
instance, as shown in FIG. 7, vibrotactile display element 
710^ located on the inside of the operator’s index finger, 
might be driven to represent forces sensed by fsr 606 1 as 
shown in FIG. 6 , on the inside of the gripper member. 
Similarly, display element 710 2 , located on the fleshy sur- 
face of the operator’s thumb, might be driven to represent 
forces sensed by fsr 606 2 , located on the inside of the 
opposite gripper member. Sensor element 710 3 , located on 
the operator’s palm, might be driven to represent forces 
sensed by fsr 606 6 , located on the outside of one gripper 
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member, with another display element, not shown, on the 
back of the user’s hand, driven to represent the force 
experienced by fsr 606 5 , on the outside of the other gripper 
member. Finally, display element 710 4 , located on the user’s 
wrist, may be driven to represent the force experienced by 5 
one or a combination of strain gauges 606 7 , 606 8 and 606 9 , 
thereby providing information about the weight of an object 
grasped by the effector. 

The choice of association between locations of fsr sensors 
on the effector and display elements used by the operator is 10 
virtually limitless. The designer should consider the sensi- 
tivity of the portion of the body contacting the display. The 
designer should also consider similarities in the conforma- 
tion of the effector and the body member, so that it is within 
the user’s capability to make the association between the 15 
display and the stimulation. The number and complexity of 
display elements must not be so great as to overload the 
user’s ability to process the information presented. 

As is mentioned above, it is helpful to vary the vibrational 
intensity of the display elements relative to the force sensed 20 
in such away that the user perceives a doubling of the 
sensation when a doubling of the force arises. Thus, a linear 
relation between force and vibrational amplitude is usually 
not warranted, as discussed in Verrillo and Chamberlain 
[1972], cited above. Rather a relation that is linear between 25 
sensation magnitude and force is preferred. 

The foregoing has discussed using a vibrotactile sensory 
modality as a substitution for force feedback in connection 
with a master slave operator. It is also possible to use the 
auditory modality as a substitution for force feedback. Such 30 
use is shown schematically with respect to FIG. 5. The 
system is the same as the system shown in FIG. 1, with 
respect to the tactile modality, except that the display is 
auditory rather than vibrotactile. Like elements in both 
systems are identified with like reference numerals. The 35 
common elements include the array of force sensing resis- 
tors 106 , the analog to digital channels 104 and the pro- 
grammable digital computer 102 equipped with input and 
output boards. An additional element is a C/MS PC music 
audio card 540 , available from Creative Labs, Inc., of San 40 
Jose, Calif. The card can be used to independently control up 
to twelve different audio signals. Each signal has a single 
square wave frequency. The card has 4 bits for amplitude 
control of a nominally right ’channel and 4 bits for ampli- 
tude control of a nominally left channel for each of the 12 45 
signals. This enables stereo presentation. There are also 
eight bits for frequency control of each of the 12 signals, and 
3 bits for octave control. The frequency and octave ranges 
produce a frequency range from 20 to 7000 Hz. This range 
contains the frequencies at which the human ear is most 50 
sensitive and the fundamental frequencies that correspond to 
all of the notes on a piano keyboard. The capabilities of the 
board allow a change in force at any of the fsr’s 106 to be 
presented to the subject with frequency and apparent loca- 
tion and/or frequency of the tone (discussed below) identi- 55 
fying the position of the force, and with amplitude (loud- 
ness) of the tone representing the magnitude of the force. 

Intensity is a physical property of sound (the rate of 
energy transfer), whereas loudness is a subjective psycho- 
physical property. Loudness is the sensation magnitude or 60 
perceived intensity of a sound, and it increases with sound 
intensity (see generally Boff, K. R., and Lincoln, J. E., 
Engineering Data Compendium: Human Perception and 
Performance . AAMRL, Wright-Patterson AFB, Ohio 
(1988)). Changes in loudness make perceptual differences to 65 
humans, and the design of the auditory loudness display 
includes the determination of a relationship to represent a 
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change in force as a change in loudness, rather than a change 
in sound intensity. This is directly analogous to the care 
taken to insure a linear relation between tactile sensor 
magnitude and force. 

Several experimental studies have been conducted to 
determine how loudness depends on stimulus intensity or 
level. Most of these studies were Conducted with a 1,000 Hz 
tone, since this is the frequency at which the ears have the 
lowest threshold when earphones are worn (see generally 
Scharf, B. and Buus, Soren, “Audition I: Stimulus, Physi- 
ology, Thresholds,” Handbook of Perception and Human 
Performance , Chapter 14. Eds. K. R. Boff, L. Kaufman, and 
J. P. Thomas. New York: John Wiley and Sons (1986)). For 
these reasons, a 1000 Hz tone is selected for the invention 
to represent forces whenever practical. However, for any 
specific operator and operator environment, other frequen- 
cies may be more useful, as one of ordinary skill in the art 
will readily appreciate. The relationship between loudness 
and sound pressure level in decibels (Scharf, B. and Hout- 
sma, A. J. M., “Audition II: Loudness, Pitch, Localization, 
Aural Distortion, Pathology,” Handbook of Perception and 
Human Performance , Chapter 15. Eds. K. R. Boff, L. 
Kaufman, and J. P. Thomas, New York: John Wiley and Sons 
(1986)) follows the relationship: 

/=0.0474* lo 00315j (8) 

where 1 represents loudness in sones and s represents sound 
pressure level in dbs, with R equal to 1.00. The value R is 
a measure of the closeness of the fit between a curve and the 
data it represents. A value for R of 1 .00 indicates the closest 
fit. The auditory signal is presented to the operator through 
speakers 520 1? 520 2 , . . . 520 x , with x running from zero. It 
is convenient to configure the speakers 520 as a pair of 
headphones, however, other arrangements with more speak- 
ers are possible. 

For instance, the operator can be surrounded with a 
number of speakers, all in the same general plane, or in 
several planes, with some speakers above and some below 
the operator. The limit on the number and placement of the 
speakers must be determined by cost and computational 
limitations, the complexity and importance of the task to be 
performed, as well as the operator’s capacity to process the 
variety of acoustic signals produced by the speakers. If more 
than two speakers are used, it is also important to locate the 
speakers in a way that corresponds to the arrangement of 
force sensors, the effector, and the effector environment. In 
the discussion that follows, the two speaker example of a 
pair of headphones is used, however, the generality of the 
invention is not so limited, and those of ordinary skill in the 
art will readily understand the means by which the apparatus 
described can be modified to accommodate numerous speak- 
ers. 

Suitable headphone speakers are available as small ear- 
plugs from Radio Shack, which fit snugly into an operator’s 
ears, having a frequency response range of sixteen Hz to 
twenty-two thousand Hz. Ear muffs, such as sold under the 
tradename Viking by the Bilsom Safety Products Company 
of Sterling, Va., may be worn over the earplugs to cut down 
on environmental noise and to increase the efficiency of the 
auditory stimulus. 

The C/MS card 540 has four bits for loudness control 
through a loudness control register. Therefore, the input 
value to the loudness control register may range from zero 
to fifteen in decimal units, where zero corresponds to zero 
loudness and fifteen corresponds to maximum loudness. The 
auditory display is calibrated with the C/MS card, and the 
relationship between the input value for the loudness control 
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register in decimal units for the C/S card and the corre- 
sponding db sound pressure level (“SPL”) output from the 
ear plugs is according to the following: 

a=1.97*c 06 (9) 5 

where a is auditory intensity (db SPL), c is input value to the 

card loudness control register, in decimals and R=1.00. 
Sound pressure level db in decibels may be related to sound 
pressure P in micropascals by the following: 

10 

db=20* log (P/20). (10) 

Converting sound pressure level from equation (9) in db to 
sound pressure level in micropascals provides: 

L=k*(P° 6 ) ( 11 ) 15 

where L is loudness in sones, k-0.01, and P is the sound 

pressure in micropascals. The sone is arbitrarily scaled and 
is defined as a unit of loudness such that a 1,000 Hz tone 
heard binaurally at 40 db has a loudness of one sone, 20 
according to the method of Scharf and Houtsma, 1986 cited 
above. 

The C/MS board 540 has four bits for volume control, i.e., 
sixteen different levels for the volume register value. Fur- 
ther, sound pressure from the board is linearly proportional 25 
to the volume register value. Therefore volume register 
value is substituted for sound pressure, P, in the above 
mentioned equation no. (11) and allowed to vary between 
zero and fifteen. Further, loudness in sones is linearly scaled 
to force between zero and five pounds for translation forces 30 
in two dimensions of a plane, designated x and z and from 
zero to ten pounds for translation forces perpendicular to the 
plane, designated in the y direction. This produces an 
equation: 

35 

F=k*(VRV ° 6 ) (12) 

where F is the force in pounds, VRV is the volume register 
value in decimal numbers, and k is a constant to be deter- 
mined and dependent on the force range. By letting zero 40 
force be represented by zero volume register value and five 
or ten pounds force (depending on the degree of freedom) be 
represented by a volume register value of 15, k is estimated 
at 0.95 for 0 to 5 pounds and at 1.97 for 0 to 10 pounds. 
These functions relate changes in the volume register value 45 
of the C/MS board, i.e., a change in sound pressure, to a 
change in the perceived force by varying the perceived 
loudness of the tone. It is then possible to develop the 
relationship that sets the loudness level of the tone based on 
the force reading on an fsr 106. This relationship is as 50 
follows, for the x and z direction forces: 

VRV (decimal)= 1 .026 * force 1 67 (13) 

where R=1.00. The relationship for the y direction forces is: ^ 

VRV (decimal)=0.3 23 * force 1 67 . (14) 

If equations (13) and (14) were used as presented, the 
volume register values would be zero (producing no sound 
at all) until the force equals 0.985 pounds in the x or z 60 
directions and 1.97 pounds in the y direction. The minimum 
force registrable by a typical fsr 106, however, is 0.01 
pounds (for the fsr’ s identified above). Thus, in order to have 
the display provide stimulation at low force levels, the 
equations (13) and (14) can be shifted to provide an auditory 65 
stimulus at lowest loudness for the minimum force measur- 
able. The following equations may be used: 


12 

VRV (decimal)=1.026*((force+0.97972) 1 ' 67 ) (15) 

and 

VRV (decimal)=0.323*(force+1.96503) 1,67 ). (16) 

At low force levels, the display may hiss, fluttering 
between on and off. To overcome this problem, a digital filter 
may be used to average a number of previous readings, for 
instance four, from an fsr 106, to determine the force value. 
Since the sampling rate is 60 Hz, this does not significantly 
slow down the effective sampling rate. 

The overall set up for a generic sensory substitution 
display for force feedback is shown schematically in FIG. 8. 
An effector 840 is equipped with fsr’s 806. The effector 
operates within an environment 860. The effector is con- 
nected by some appropriate channel, indicated by dotted 
lines 842, to a control device 802, such as a properly 
programmed digital computer. The fsr’s 806 are also con- 
nected through a channel, indicated by dotted lines 844, to 
a control device, which may conveniently be the same 
control device 802 as controls the effector 840, or it may be 
another control device more specially tailored to the control 
of the fsr’s. 

The channel may be a plurality of electrical wires, or fiber 
optic cables. Alternatively, the channel may be a radio 
frequency (or light frequency, i.e. laser) channel, for 
instance as may be used to communicate with an effector 
located on a satellite in outer space, controlled by an 
operator on earth. 

Visual feedback is provided by image capture device 852, 
for instance a video camera, connected through an appro- 
priate channel to an image display device 854, such as a 
cathode ray tube, located in the local environment of the 
operator. 

At the local environment of the operator, the sensory 
substitution display 812 provides force feedback through a 
modality other than reactive force, such as vibrotactile or 
auditory. The display elements 870 are applied to the opera- 
tor in the appropriate fashion, depending on their modality. 
They are under the control of a control device 802, again 
conveniently embodied in a digital computer. The operator 
provides manual input signals to an input device 820, such 
as a levered joystick or an actuator glove, which generates 
signals that are used by the control device that controls the 
effector 840, such as digital computer 802. 

Use of sensory substitution displays has proved beneficial 
in certain circumstances. Evaluation of the success of such 
use may be considered in light of certain criteria. 

For instance, one measure is the number of absolute 
judgments that the user can make based on the substituted 
stimuli. Four test subjects each began by trying to distin- 
guish among three different frequencies (or, in a separate 
test, amplitudes). As the session continued, the number of 
stimuli in a grouping were increased to four, five, six, seven, 
and ten different stimuli. Each subject received each stimu- 
lus within a group four times for a total of sixteen x n trials 
per group, where n was the number of stimuli within a 
group. The subjects were trained and given practice in 
recognizing different frequencies (or amplitudes) and 
assigning numbers to each stimulus in a group. Each stimu- 
lus was presented for 2.5 seconds. The subject then 
responded by assigning a number to that stimulus. After 
responding, the subject was told whether or not the response 
was correct, and if it was incorrect the subject was told the 
correct response. This process continued until all trials for 
that group was completed. Before each group of tones was 
presented, the subject was given some warm-up and practice 
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trials in order to more easily adjust to the increased number 
of stimuli in the new group. 

The inventor conducted channel capacity experiments 
using auditory stimuli varying in loudness to test the audi- 
tory display. The stimuli were spaced by equal sensation 
magnitude based on the sone scale (see above) between 46 
to 70 db SPL. A 1000 Hz auditory tone was presented to both 
ears simultaneously. A channel capacity of approximately 
1.76 bits representing approximately 3.4 absolute judgments 
was determined. 

Auditory frequency may also be used to provide infor- 
mation about force feedback, particularly when a force can 
be applied to the effector in more than one direction. Test 
subjects were presented with a number of tones varying in 
frequency and were asked to assign a number to each tone. 
The number of tones (different pitches) were increased until 
the channel capacity was determined. The stimuli were 
spaced equally on a logarithmic scale from 28 to 3500 Hz. 
The tones were simultaneously presented to both ears at an 
intensity of approximately 65 db spl. The results show a 
channel capacity of approximately 2.18 bits or 4.5 absolute 
judgments. 

Similar amplitude experiments have been conducted with 
the vibrotactile display discussed above. The vibrotactile 
display allowed amplitude modulation of a 250 Hz vibratory 
signal with an amplitude range of 7 to 32 db (i.e., 1 micron). 
This range was divided into sections of equal sensation 
magnitude for the absolute judgments experiments. The 250 
Hz vibratory signal was presented simultaneously to the 
index finger and thumb. A channel capacity of approxi- 
mately 1.76 bits representing approximately 3.4 absolute 
judgments was determined. 

In many teleoperation tasks an operator needs to distin- 
guish whether an applied force is increasing or decreasing 
over time. In order to calibrate the effectiveness of the 
vibrotactile and auditory displays to present this informa- 
tion, tests on relative discrimination to determine just notice- 
able differences were conducted. These tests determined the 
operator’s ability to detect a small difference between two 
stimuli or a change in one stimulus. The least change in a 
stimulus or the least difference between two stimuli that can 
be detected is called the Just Noticeable Difference (JND) 
which is also expressed as AI to represent a change in 
intensity. 

The C/MS card 540 had four bits for volume control, 
providing fifteen discrete loudness levels between 46 and 70 
db SPL. Therefore a maximum of 14 JND’s were possible. 
These levels were tested and all fifteen levels were easily 
identifiable when presented relatively from one to the next, 
thus providing 14 JND’s for the auditory loudness display. 

Similar experiments conducted with the vibrotactile dis- 
play show approximately fifteen JND’s within the range of 
zero to forty microns. To some extent, the noticeability of a 
difference depends on the location along the amplitude 
spectrum. See generally Stevens, S. S., “Mathematics, Mea- 
surement, and Psychophysics,” S. S. Stevens (Ed.), Hand- 
book of Experimental Psychology. New York: John Wiley & 
Sons (1951) and Rabinowitz, W. M., Lim, J. S., Braida, L. 
D., and Durlach, N. L, “Intensity Perception VI: Summary 
of Recent Data on Deviations from Weber’s Law for 1000 
Hz Tone Pulses,” Journal of the Acoustical Society of 
America , Vol. 59, No. 6, June, 1976, pp. 1506-1509 (1976). 

Experiments conducted by the inventor have shown, in 
general, that for most applications, conventional reactive 
force feedback allows the operator to accomplish tasks more 
quickly or more precisely than does the use of either 
auditory or vibrotactile sensory substitution force feedback. 
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However, as has been mentioned above, reactive force 
feedback is expensive, and adds to the bulk, weight and 
complexity of systems requiring its use. Thus, for instances 
where a tradeoff is acceptable between speed or accuracy on 
5 the one hand, and cost, bulk, weight, complexity, etc. on the 
other hand, sensory substitution according to the present 
invention provides distinct advantages. 

Vibrotactile and auditory sensory substitution force feed- 
back show a distinct advantage over conventional reactive 
10 force feedback in the presence of a time delay, such as would 
be present in connection with control of an effector in outer 
space. As is well known, and mentioned above, use of 
reactive force feedback is impossible in the presence of a 
time delay because instabilities arise. The instabilities arise 
15 because the reactive feedback is applied to the operator’s 
body, typically the hand, which is manipulating the input 
device. Thus the time delayed force feedback itself becomes 
an input to the system, quickly resulting in instabilities. 
Neither the auditory nor the vibrotactile force feedback 
20 signals result in a direct input signal to the operator’s input 
device. Thus, the instabilities do not arise. 

The vibrotactile and auditory modalities of force feedback 
are particularly useful in those situations where visual 
feedback, which is common in such systems, is unavailable, 
25 or impaired or otherwise degraded. Such situations would 
include underwater operations, or operations in dirty atmo- 
spheres, or operations where physical portions of the envi- 
ronment simply block the view of the image capture device 
from time to time. Moreover, high fidelity video feedback 
30 requires a significant signal bandwidth. Use of the sensory 
substitution according to the invention permits use of lower 
fidelity video feedback, requiring a smaller bandwidth. 

Another reason that vibrotactile and auditory sensory 
substitution force feedback have an advantage over reactive 
35 force feedback is that lower force thresholds are detectable 
using these substituting modalities. This is because very 
small force signals can be amplified using the control 
systems, without affecting the operator’s ability to provide 
an input signal to the system. With a reactive feedback 
40 system, if very small forces are amplified so that the 
feedback signal is noticeable to the operator, this necessarily 
means that larger forces will also be amplified, thus, making 
it more difficult for the operator to move the input device in 
the presence of these large forces. Thus, for simple tasks 
45 where all that is required is to know whether or not contact 
between the effector and the environment has been made, 
vibrotactile or auditory sensory substitution force feedback 
may provide operational advantages over reactive force 
feedback, even in the absence of a time delay and with 
50 unimpaired visual feedback. 

An additional advantage of the sensory substitution 
modalities of force feedback is that they do not invoke in the 
operator’s hand or arm (or other limb implementing control) 
a reaction. Thus, the operator can more easily control the 
55 input device, without needing to compensate for the reaction 
to the force feedback. 

There are many applications in which increased sensitiv- 
ity of force detection could be useful. For example, if a 
physician were performing a surgical procedure in which 
60 knowing the amplitudes of small forces being applied to the 
patient were helpful, sensory substitution could provide 
advantages. For a small force applied to the patient, an above 
threshold stimulus could be presented through sensory sub- 
stitution. If the same technique were used with a high 
65 feedback gain for traditional force feedback, movements 
could be slowed or made overly difficult. High sensitivity to 
small forces could also be useful for a telemanipulation task 
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involving a delicate instruments or one that requires the 
application of small forces for successful task completion. 

The mode of implementation of a sensory substitution 
display depends upon the environment, effector, input 
device and other criteria. Considering first an auditory 5 
modality, numerous fsr’s can be located at various locations 
on the effector. If more than two speakers are used, such as 
a special room fitted with many speakers at many planes 
relative to the operator, each fsr can be associated with a 
single speaker, or a combination of speakers, thereby giving 10 
a direct impression of the location of the force. The ampli- 
tude of the auditory signal can be used to indicate the 
amplitude of the force. Alternatively, if frequency is not used 
for any other indication, varying frequency can be used to 
indicate force amplitude. 15 

Even with only a pair of speakers, such as headphones, 
signals from various fsr’s spread out essentially in a plane 
can be mixed and sent to the pair of speakers to give the user 
a more complicated spatial presentation. This would be 
much as a pair of headphones or speakers can convey to a 20 
listener the locations of musical instruments spread out over 
a stage in a symphonic presentation. Frequency can be used 
to indicate the direction of forces applied other than in a 
single plane. For instance, higher frequency can indicate 
force being applied from above a reference plane, while 25 
lower frequency can indicate force being applied from a 
direction below the reference plane. The frequency cue may 
be used either alone or in combination with a degree of the 
spatial representation similar to symphonic representation 
discussed above. 30 

Considering the vibrotactile modality, an example has 
been shown with reference to FIG. 7 where a mapping is 
made between the operator 5 s hand and the elements of the 
effector, as discussed above. With differently shaped effec- 
tors, different numbers of sensors can be used, in different 35 
locations. For a different sort of effector, for instance a robot 
walking on two legs, vibrotactile displays may be connected 
to the operator’s feet and/or legs, rather than the hand. Both 
hands or feet can be used. Alternatively, an array of display 
elements can be worn by the user, such as in a waist belt or 40 
a headband, providing 360° of direction information in a 
single plane, very similar to the type of information that 
would be provided through a similar array of audio speakers. 

Use of the vibrotactile display could be made by an operator 
with hearing or seeing impairments. 45 

The invention may also be used to provide force feedback 
for the user of an artificial limb, such as a hand, gripper or 
foot. Taking an artificial hand as an example, force sensing 
sensors would reside in the fingertips, and other locations, 
such as the wrist. The force feedback would then be dis- 50 
played, for instance, through vibrotactile displays on the 
patients skin. The display location could be on the stump of 
the limb involved, the opposite limb, if available, or other 
locations. This information could be useful for common 
tasks such as grasping delicate objects, opening doors, or 55 
finding objects in the dark. 

Some of the work underlying the present invention was 
discussed in Massimino, M. J. and Sheridan, T. B., “Sensory 
Substitution of Force Feedback For the Human-Machine 
Interface in Space Teleoperation,” delivered at the World 60 
Space Congress in Washington, D.C., Aug. 31, 1992, avail- 
able from the American Institute of Aeronautics and Astro- 
nautics c/o Tasco, 9 Jay Court, P.O. Box 753 Waldorf, Md. 
20604, as paper No. IAF/IAA-92-0246, which is incorpo- 
rated fully herein by reference. The invention is also 65 
described in Massimino, M. J., “Sensory Substitution For 
Force Feedback in Space Teleoperation,” submitted in par- 
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tial fulfillment of the requirements for the degree of Doctor 
of Philosophy in Mechanical Engineering at the Massachu- 
setts Institute of Technology, June 1992, which is also fully 
incorporated herein by reference. 

The foregoing discussion should be understood as illus- 
trative and should not be considered to be limiting in any 
sense. While this invention has been particularly shown and 
described with references to preferred embodiments thereof, 
it will be understood by those skilled in the art that various 
changes in form and details may be made therein without 
departing from the spirit and scope of the invention as 
defined by the claims. 

Having described the invention, what is claimed is: 

1. An apparatus for an operator to control an effector that 
is remote from the operator to interact with an environment 
that is also remote from the operator by operator manipu- 
lation of an input device in an environment local to the 
operator, said apparatus comprising: 

a. at least one means for sensing an amplitude of at least 
one force arising between the remote effector and the 
remote environment; 

b. means for generating a force feedback signal that 
corresponds to the amplitude of said sensed force; 

c. means for transmitting said force feedback signal from 
said remote environment to the location of said opera- 
tor; 

d. means for transducing said force feedback signal into 
a vibrotactile sensory substitution signal to which the 
operator is sensitive; and 

e. means for presenting said transduced sensory substitu- 
tion signal to a vibrotactile system of the operator that 
is receptive to said vibrotactile sensory substitution 
signal such that a full range of force amplitude may be 
represented by said means for presenting said trans- 
duced vibrotactile sensory substitution signal. 

2. The apparatus of claim 1, said means for transducing 
comprising means for representing variations of a prese- 
lected component of said force feedback signal by variations 
of an amplitude of said vibrotactile sensory substitution 
signal. 

3. The apparatus of claim 1, said means for transducing 
comprising means for representing variations of a prese- 
lected component of said force feedback signal by variations 
of a frequency of said vibrotactile sensory substitution 
signal. 

4. The apparatus of claim 1, said means for presenting 
said transduced sensory substitution signal comprising 
means for presenting said vibration signal to a plurality of 
different receptive locations on the operator, each location 
corresponding to a different of said at least one forces arising 
between the remote effector and the remote environment. 

5. The apparatus of claim 1, said means for presenting 
said transduced sensory substitution signal comprising at 
least one mechanical vibrator. 

6. The apparatus of claim 5, said means for presenting 
said vibration signal comprising means for attaching said at 
least one vibrator to the operator’s hand. 

7. The apparatus of claim 5, said means for presenting 
said vibration signal comprising means for attaching said at 
least one vibrator to the operator’s head. 

8. The apparatus of claim 6, said means for attaching said 
at least one vibrator to the operator’s hand comprising 
means for attaching said at least one vibrator to at least one 
of the operator’s fingers. 

9. The apparatus of claim 6, said means for attaching said 
at least one vibrator to the operator’s hand comprising 
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means for attaching said at least one vibrator to the opera- 
tor’s wrist. 

10. The apparatus of claim 5, said means for presenting 

said vibration signal comprising means for attaching said at 
least one vibrator to the operator’s waist. 5 

11. An apparatus for an operator to control an effector that 

is remote from the operator to interact with an environment 
that is also remote form the operator by operator manipu- 
lation of an input device in an environment local to the 
operator, said apparatus comprising: 10 

a. with respect to at least one force arising between the 
remote effector and the remote environment, said force 
acting along a direction, at least one means for sensing 
said direction of said at least one force; 

b. means for generating a force feedback signal that 15 
corresponds to the direction along which said sensed 
force acts, said feedback signal being characterized by 

at least one component having a frequency and ampli- 
tude; 

20 

c. means for transmitting said force feedback signal from 
said remote environment to the location of said opera- 
tor; 

d. means for transducing said force feedback signal into 

a vibrotactile sensory substitution signal to which the 2 5 
operator is sensitive; and 

e. means for presenting said transduced vibrotactile sen- 

sory substitution signal to a vibrotactile sensory system 
of the operator that is receptive to said sensory substi- 
tution signal. 30 

12. The apparatus of claim 11, said means for transducing 

comprising means for representing variations of a prese- 
lected component of said force feedback signal by variations 
of an amplitude of the vibrotactile sensory substitution 
signal. 35 

13. The apparatus of claim 11, said means for transducing 
comprising means for representing variations of a prese- 
lected component of said force feedback signal by variations 
of a frequency of the vibrotactile sensory substitution signal. 

14. The apparatus of claim 11, said means for presenting 40 
said transduced vibrotactile sensory substitution signal com- 
prising means for presenting said vibration signal to a 
plurality of different receptive locations on the operator, 
each location corresponding to a different of said at least one 
forces arising between the remote effector and the remote 45 
environment. 

15. The apparatus of claim 11, said means for presenting 
said transduced vibrotactile sensory substitution signal com- 
prising at least one mechanical vibrator. 

16. The apparatus of claim 15, said means for presenting 50 
said vibrotactile sensory substitution signal comprising 
means for attaching said at least one vibrator to the opera- 
tor’s hand. 

17. The apparatus of claim 15, said means for presenting 
said vibrotactile sensory substitution signal comprising 55 
means for attaching said at least one vibrator to the opera- 
tor’s head. 

18. The apparatus of claim 16, said means for attaching 
said at least one vibrator to the operator’s hand comprising 
means for attaching said at least one vibrator to at least one 60 
of the operator’s fingers. 

19. The apparatus of claim 16, said means for attaching 

said at least one vibrator to the operator’s hand comprising 
means for attaching said at least one vibrator to the opera- 
tor’s wrist. 65 

20. The apparatus of claim 16, said means for attaching 
said at least one vibrator to the operator’s hand comprising 
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means for attaching said at last one vibrator to the operator’s 
palm. 

21. The apparatus of claim 15, said means for presenting 
said vibration signal comprising means for attaching said at 
least one vibrator to the operator’s waist. 

22. The apparatus of claim 12, said vibrotactile sensory 
substitution signal being characterized by at least two com- 
ponents, each having a frequency and an amplitude, said 
means for representing variations of a preselected compo- 
nent of said force feedback signal by variations of an 
amplitude of the vibrotactile sensory substitution signal 
comprising means for combining variations of the ampli- 
tudes of said at least two components of the vibrotactile 
sensory substitution signal. 

23. An apparatus for an operator to control an effector that 
is remote from the operator to interact with an environment 
that is also remote from the operator by operator manipu- 
lation of an input device in an environment local to the 
operator, said apparatus comprising: 

a. with respect to at least one force arising between the 
remote effector and the remote environment, said force 
acting at a location of the effector, at least one means 
for sensing said location at which said force acts; 

b. means for generating a force feedback signal that 
corresponds to the location of said sensed force; 

c. means for transmitting said force feedback signal from 
said remote environment to the location of said opera- 
tor; 

d. means for transducing said force feedback signal into 
a vibrotactile sensory substitution signal to which the 
operator is sensitive; and 

e. means for presenting said transduced vibrotactile sen- 
sory substitution signal to a vibrotactile system of the 
operator that is receptive to said sensory substitution 
signal. 

24. An apparatus for an operator having a vibrotactile 
system to receive information regarding an environment 
remote from the operator, said apparatus comprising: 

a. an input device located in an environment local to the 
operator; 

b. an effector located in the remote environment; 

c. means for generating a signal in response to manipu- 
lation of the input device; 

d. means for manipulating said effector in response to said 
signal representative of manipulation of the input 
device; 

e. at least one means for sensing a force arising between 
the effector and the remote environment; 

f. means for generating a force feedback signal that 
corresponds to said sensed force; 

g. means for transmitting said force feedback signal from 
said remote environment to the location of said opera- 
tor; 

h. means for transducing said force feedback signal into 
a vibrotactile sensory substitution signal to which the 
operator is sensitive; and 

i. means for presenting said transduced sensory substitu- 
tion signal to a vibrotactile system of the operator that 
is receptive to said vibrotactile sensory substitution 
signal. 
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